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5G : ity against the dynamic nature of the mobile network, such as load variations and link 
Free-space optical failures. The proposed solution offers changing propagation medium between free- 
mm-Wave space optical (FSO) and radio frequency (RF); this dynamic change between every two 
Route selection hops reduces power consumption, increases throughput, and minimizes latency. Based 
Self-backhauling on the performed simulation, our proposed algorithm measured better overall bit er- 

ror rate (BER) compared to both RF-only and free-space optical FSO-only techniques 

resulting in increased backhauling capacity and reduced overall route interference. 
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1. INTRODUCTION 

Today, many researchers show more interest in the cellular network, specifically those focusing on the 
millimeter-wave bands (mm-Wave) as a door to 5G networks. Such mm-Wave bands have a limitation in the 
communication distance for their short communication range due to high path loss and very high penetration 
loss through different obstacles such as the human body and buildings [1], [2]. To deal with such constraints, 
an ultra-dense deployment becomes essential for mm-Wave cellular networks to reduce the effect of these 
constraints [1], [2]. 

Furthermore, the backhaul in the mobile network connects the radio frequency (RF) tower at the cell 
sites to the corresponding core network, ensuring that the mobile users' connectivity is available. Normal 
backhauling in the current architecture is unscalable and it is not cost-effective to have a dedicated fiber- 
optic link connected to each base-station (BS) for mobile operators and having more locations to deploy these 
stationary base-stations. To find a solution for ultra-dense deployment, researchers are investigating the concept 
of self-backhauling [3]. In self-backhauling, the fiber-optic links connected to the core network are associated 
with a fewer number of selected base-stations (we refer to them as Fiber-BS in this work). The remaining base- 
stations are connected to the proposed Fiber-BS using wireless links operating using the industry-standard 
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wireless communication protocols. 

The self-backhauling concept is defined in wireless communication by establishing backhaul links, 
which comprise all the intermediate network links (both wired and wireless connections links) between cell 
sites at the network edge with the corresponding core network. Besides, in self-backhauling, all the com- 
munication between the neighbor base-stations shares the same channels and spectrum as the communication 
between the mobile users and the corresponding stationary base-stations. Using self-backhauling leads to in- 
creased network capacity while keeping the installation and operational costs at the minimum. 

The 5G mobile network is a new standard coming after the 4G network in the telecommunication 
industry to enable an unlimited number of devices and objects to be connected. It is using a wider spectrum 
such as sub-6 GHz and mm-Wave. The reason behind developing this technology is the need for a very high 
data communication speed, which could be up to multi-Gbps data rate, very low end-to-end latency with an 
ultra-high capacity to enable massive networks. 

Every new technology comes with new challenges. For the 5G technology, the challenge arises from 
the underlying nature of the used technology mm-Wave, where it is suffering from high path loss of signal 
with high penetration loss, which results in making the signal affected by both dynamic and static types of 
blockage. To minimize these effects, it needs a densification deployment to ensure a sufficient level of an al- 
ternative route to access the core network through wireless communication between the available base stations. 
Figure[I]illustrates the self-backhauling comparing to fronthauling. 


Wired Backhauling Link 


Figure 1. Self-backhauling and fronthauling 


Challenges such as interference, computational complexity, quality-of-service (QoS), and robustness 
are present when the self-backhauled mm-Wave solution is deployed. Such challenges appear because of the 
high path lose of the mm-Wave and the need for ultra-dense deployment. It is essential for a self-backhauling 
system to use proper routing and scheduling algorithms. Using a routing algorithm that is designed for another 
environment may cause some problems. For example, using the shortest path algorithm may not be suitable for 
a self-backhauled system due to: i) huge number of users arrivals causing a packet drop and buffer overflow 
(e.g., a bus is crossing the cell), ii) temporary blockage because of moving objects (e.g., large vehicles or a 
mobile user crosses behind a large building), and iii) first and second-line interference from the surrounding 
base stations. One of the main components in the proposed system architecture is using unmanned aerial 
vehicle (UAV) as a self-backhualing base-station (S-BS). Furthermore, we introduce a new notion to enhance 
the overall system performance by mixing RF with free-space-optical (FSO) propagation media in an adynamic 
way while considering the different types of fading between every two consecutive hops in the selected route. 

In this study, we addressed self-backhauling routing challenges in high altitude UAV networks with 
adaptive communication medium selection between FSO and RF. Due to the use of mm-Wave RF communi- 
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cation systems, many factors affect the choice of which medium to use in data transmission. For example, 
temporary obstacle blockage, pointing errors, and weather conditions are some of the factors that affect the 
decision of which medium to use. Even though the impact of these factors might be temporary [4], but their 
frequent occurrence might affect the cost of the selected route significantly in case the route is not adapted to 
mitigate these challenges. 

The contribution of this work is to develop a routing scheme. We determine the route the data should 
take based on calculating the links’ weight and selecting the appropriate path. The weight of the medium is 
based on the signal-to-noise ratio (SNR) value of each of the available paths. For RF medium, the SNR is 
modeled by Rayleigh and Gamma-Gamma distributions, while for FSO links, the SNR is measured taking into 
consideration the atmospheric disturbance and the pointing error fading across the available paths. 

Our model is based on a scalable algorithm that uses a semi-distributed approach with artificial intel- 
ligence (AI) learning agents attached to every S-BS to provide scalability to the solution, taking into consid- 
eration the latency, interference, and traffic variation as well as the channel fluctuations. Within the proposed 
solution, we are using UAV as an S-BS rather than the commonly used RF base stations (BSs) while keeping 
the RF stationary BSs as a front hauling BSs. Furthermore, we are providing a hybrid selection mechanism to 
utilize the FSO over RF whenever it is possible. 

Our proposed solution consists of two types of multi-level S-BS cellular network BSs: Fiber-BS as 
backhauled BSs (F-BSs) and self-backhauled BSs (S-BSs). Regarding F-BS, it is a stationary backhauling 
tower connected to the core network via a dedicated fiber-optic link, where the UAV S-BSs reach it via multi- 
hop wireless links, either using FSO or through RF communication based on the measured SNR value between 
every two hopes (UAV S-BS) on the selected route and the overall routes selection is based on the average bit 
error rate (BER). Each BS is equipped with a single RF and a single FSO interface. Within this paper, we are 
assuming that all RF communication is carried using the same RF bands. To evaluate the proposed solution, we 
rely on two metrics: first, we will evaluate the proposed algorithms by examining the computational complexity 
of each of them and calculate the overall performance using Big O notation (O(f(n))). Then, we will perform 
an extensive simulation to illustrate the reliability and robustness in different situations; and finally, we will 
compare the solution performance with a selected number of well-known techniques such as SCAROS, RT, 
and MTFS algorithms. 

The rest of this paper is organized: section 2 shows the background and related work to this topic. 
Section 3 provides an overview of our proposed algorithm. Performance evaluation, analysis, obtained results, 
and a comprehensive discussion is presented in section 4. Finally, we give our conclusions in sections 5 


2. RELATED WORK 

As the world and telecommunication operators go beyond 4G and 5G networks, their objectives and 
goals are driven by their customers' needs. Mobile customers become in need of higher network capacity, 
bandwidth, throughput improvement, low latency, interference minimization, and decreasing in the measured 
BER to improve the QoS. Achieving these improvements requires new concepts and techniques to emerge 
with a great improvement in the network architecture itself. An example of these techniques is the ultra-dense 
deployment and mm-Wave utilization, which are considered as a prominent solution for these architectural 
improvements demands. To proceed in this paper, it is essential to give an overview of each of these used 
technologies and techniques. 


2.1. Ultra-dense networks for 5G networks and beyond 

To cope with the expected increase in the data traffic and due to the number of connected users as 
well the nature of the used applications, one of the main aspects that have to be taken into consideration is 
infrastructure densification. One of the main techniques to achieve infrastructure densification is ultra-dense 
network deployment [5]. The ultra-dense network deployment plays a significant role in this goal; it means that 
a large number of small cells have to be deployed to form a robust 5G network. But going with this technique 
potential will raise several new challenges related to mobility, backhauling, and interference. Table[I|shows an 
example of different dense deployment per cellular network. 

To overcome these challenges, a new layer of network functionalities need to be implemented hier- 
archically. Besides that, the researchers in this field of study try to minimize the interference using different 
techniques such as inter-cell interference coordination or dynamic carrier selection in an autonomous way. All 
of the mentioned techniques are applicable in small cells environment, but still, it is prone to flexibility failure 
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[6] which makes them not suitable in the 5G and beyond networks; these networks need more flexibility, scala- 
bility, and lower interference [6]. For these types of challenges, a new emerging concept shows great potential 
in increasing the data rate while reducing the interference; this technique is called self-backhauling. 


Table 1. Ultra dense in 5G cellular networks 
Cellular network — Number of base stations 


3G 4 — 5/ (km)? 
4G 8 — 10/(km)? 
5G 40 — 50/ (km)? 


2.2. Self-backhauling for 5G networks and beyond 

Due to the increasing demand for data traffic, using mm-Wave becomes essential to enhance the data 
rate and throughput. However, when intended to use these frequency ranges, more system design challenges 
come to the surface, showing more need for network densification with efficient cost deployment. To figure 
out all these challenges, the concept of self-backhauling in cellular communication shows a promising result 
especially by increasing the network capacity while reducing the deployment cost [7]. The self-backhauling 
technique is defined in wireless communication between a BS and the next BS sharing the same resource that 
is already used between mobile users and their corresponding base station until reaching the core network. 
The main advantages of using this technique are increasing network capacity and reducing installation and 
operation costs. 


2.2.1. Why self-backhauling? 

When asking why we need the self-backhauling, the simple and straightforward answer is the high 
density of the deployed network to achieve better performance, increase the availability, and increase the scal- 
ability of the available backhaul links. Self-backhauling in a cellular network is the enabler of reducing the 
reliance on the wired backhaul for every deployed BS and the access node. As a result, it gives a simple and 
cost-effective deployment solution with better network planning and minimal deployment and operation efforts. 

The advantages of using the self-backhauling can be summarized in the pointS;: i) better spectrum 
reuse efficiency between different BS and backhauling link, ii) cost efficiency resulting from sharing the same 
hardware. Which eventually result in reducing the running operation and maintenance cost, and iii) minimize 
the corresponding latency, which usually affects the transmission. 

Utilizing the self-backhauling technique is a promising technique to solve most of the facing chal- 
lenges. However, still, some challenges will be left unresolved especially that we are using mm-Wave. One 
of these challenges is the blockage percentage due to high penetration and path loss of the mm-Wave and the 
interference with the other self-backhauling BS. To overcome these challenges, the use of an UAV as a S-BS 
becomes important. 


2.3. UAV as self-backhauling base station (S-BS) 

Utilizing the UAVs in the telecommunication industry shows promising results in providing a solu- 
tion for a large number of challenges, especially beyond 5G era. This utilization can help in increasing the 
backhauling network capacity with relatively low cost in both mobile and stationary users. In addition, UAV 
utilization will provide a high level of network flexibility in terms of coverage area and latency. 


2.3.1. Why utilizing UAV as a S-BS? 

When digging in this area, you can find many related works where they focus on using the UAV in 
three different ways [2], [8], and [9]. The first one is using the UAV as a normal base station and allowing 
mobile users to connect directly, and passing their traffic between these users and the core network directly 
Figure 2(a), and this is the most common focus. We believe that this technique can be useful in temporary events 
like frequent temporary blockage resulting from user or object mobility or by load amplification from a huge 
or sudden arrival of many users, which could interfere with the neighboring cell when trying to communicate 
ground and aerial BS. The second focused area is using the UAV as front hauling where it acts as a relay 
between the mobile users and another stationary base station Figure 2(b), this approach decreases the blockage 
percentage, but still, the resulting interference from a neighboring cell is left unresolved. 

The last approach which we used and utilized in our study is using the UAVs as a backhauling hub con- 
nected to the core network Figure 2(c); our contribution to this approach is using the UAV as a self-backhauling 
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in a multi-hoping and multi-hierarchical way to deliver the transmitted traffic by the mobile users through the 
connected stationary base station (small, macro or micro cells) to and from the corresponding wired fiber core 
network (F-BS). Using the UAV as S-BS is a promising solution to solve many of the facing problems such as 
reducing blockage and interference. Still, a new number of challenges could appear especially those related to 
the UAVs. An example of these challenges is the battery limitation capacity. To overcome these challenges, we 
are proposing a hybrid transmitting approach by utilizing FSO and RF for dynamically medium selection for 
mobile user traffic transmission. 
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Figure 2. UAV as BS approaches (a) UAV as normal base station, (b) UAV as front hauling, and (c) UAV as 
backhauling 


2.4. Hybrid FSO/RF communication medium 

During the last years, the researcher shows that using FSO as a communication medium has a great 
potential and promising solution for many RF common challenges. The researcher in this area defines FSO as a 
LOS technology that uses a light-emitting diode (LED) or laser as a communication medium to provide a data 
rate in Gbps. The transmission of this optical communication is done in the free space as the term describes it 
(free space optical). 

Based on the previous studies, FSO can provide a data rate up to 10 Gbps without any need for 
any wiring solution (e.g., fiber-optic cabling) or spectrum licensing like radio frequency bands. As a benefit 
over the normal wiring fiber optics, FSO has less installation cost, maintenance cost, more flexibility, and 
better bandwidth efficiency since the light travels in glass (fiber optical cable) is slower than traveling in the 
free space. So we can fairly say that the FSO can transmit the data, voice, and video at the speed of light. 

To utilize the FSO technology in telecommunications, we need three types of components, a transmit- 
ter, a medium, and a receiver. The first component is the transmitter, which is used to modulate and transmit 
the optical signal. The second component is the medium channel (free space), which carries the signal (this 
eventually suffers from signal attenuation resulting from the weather turbulence such as fog, haze, cloud, tem- 
perature, and rain). The last component is the receiver, which responsible to re-modulate the transmitted signal. 
One of the main advantages of using FSO (in addition to the great bandwidth) is the long transmission distance 
which reaching up to 5 km (in some circumstances, this distance could reach up to 12 km) comparing to the 
mm-Wave, which is not effective for distances exceed 150 meters. Also, the needed consumption power for 
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the transmission (which is a critical point in this research as we utilize UAVs as a flying base station with a 
relatively limited battery capacity) in the mm-Wave is higher than the FSO. 


2.5. Literature review 

One of the newest techniques using with mm-Wave is the self-backhauling, which optimize the net- 
work usage and increase its efficiency. These techniques are proven to be an NP-hard problem in its simplest 
form without considering the interference and the mobility nature of the network [1], [11]. Unfortunately, this 
could work by simplifying and relaxing the problem by adding some assumptions to get non-NP-hard formu- 
lations [2], [11], or looking for heuristics to simplify the problem constraints to build an incomplete or 
non-optimal solution to the mentioned problem [1]. Some of these assumptions and constraints simplification 
are the interference-free assumption in [I], to find a mathematically tractable goals function. In reality, the 
mm-Wave networks are suffering from interference because of the deficiency of it is beamforming [9], as 
wells as the first and second-order reflections especially inside the crowded cities [2], (8]. 

Besides that, in (1]-/[3], (13]-[I5] are assuming that there will be a fully-backlogged queue without 
considering the change of the traffic or the imbalance across the different segment inside the network, which 
could cause bottlenecks that decrease the performance, exhausting the resources and resulting in QoS degra- 
dation. Moreover, most of the proposed solutions are trying to maximize the throughput [1], (2], C0-03) 
without considering the latency in the backhaul links as an important factor. Because of the critical need for 
adaptability in these systems, most of the research focused on how to mitigate the resulting link outages from 
the beam misalignment due to the user mobility or links blockage [16]-[[18]. Some authors like [16], and 
(19], proposed algorithms to predict the blockages and select the best alignment of the beams by analyzing 
users’ mobility [16], [17]-([20]. 

Yuan et al. tried to solve the route selection and scheduling problem as a linear programming 
problem to find the optimal throughput using the matching theory; Yuan et al. in relaxed their problem 
by eliminating the interference factor. Till now, and based on our research and analyzing a lot of prior works, 
most of them do not take into account the interference, latency, first/second reflection, blockage, and changing 
of the traffic load in the same study. Moreover, many authors did not provide adaptability and robustness as a 
feature inside their solution while using dynamics networks, leading to link failures, frequent and unexpected 
changes in the traffic and channel fluctuations. Such issues make the network need to recompute the overall 
solution and select a new route, and rescheduling the traffic accordingly in case of any change. The work of 
is the closest to our model, where the authors eliminate the interference-free assumption and take into account 
the latency, interference, and traffic load and manage the adaptability nature and robustness networks dynamic 
such as link failure, traffic variation and channel fluctuations. In addition, the authors focused on the solution 
(solution-oriented modeling) rather than focusing on the root of the problem (causal modeling), where they 
focus on reducing the major impact variables. They come with so-called symptom-based modeling in which 
they formulate the problem based on the symptom instead of what caused them. But in their solution, they are 
not trying to focus on solving the blockage which could introduce more interference. For their learning agent, 
they intended to use a greedy-based algorithm to return the optimal value. In contrast, this algorithm returns an 
optimal value but sometimes could go in an infinite loop. 


3. PROPOSED SOLUTION 

The solution we are proposing is a variation of the scalable and robust self-backhauling solution for 
highly dynamic millimeter-wave networks (SCAROS) algorithm [4], in which we replace RF-BSs with high 
altitude S-BSs through the use of UAVs. Reductions in the blockage percentile and the interference probability 
are expected through the implementation of our proposed solution. We also adapt the data transmission medium 
to be highly dynamic and robust by accommodating the FSO technology based on the measured environment 
situation, as well as the regular RF communication. Our proposed solution consists of two types of multi- 
level UAV S-BS cellular network base-stations, i) Fiber-BS acting as backhauling base stations (F-BS) and 
ii) UAV-BS acting as self-backhauling base stations (S-BS). 

F-BS is a regular stationary backhauling tower connected to the core network through a dedicated 
fiber-optic link. On the other hand, S-BS is a UAV-based self-backhauling base-stations that communicates 
with the F-BS through multi-hoping wireless links. These links could be either FSO or RF link based on the 
measured bit error rate (BER) value between any two hops. A graphical representation of the proposed solution 
is provided in FigureB] Each of these base-stations is equipped with an FSO interface in addition to the regular 
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RF interface. In the proposed solution, we are assuming that all RF communication is done using the same RF 


bands [21]. 
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Figure 3. Self-backhauled mm-Wave UAV based cellular networks 


To have an efficient communication system, both the allocation of time-slots and the route selection 
issues have to be addressed. Regarding the time-slotting issues, we are assuming that all time slots within our 
system have equal duration. Every time slot within it is denoted by a time index (T), where T is between | and 
a maximum value (t). As for the routing issue, we defined it for time slot t as X; € X, where X represents all 
the feasible route selection and scheduling solutions. Specifically, X; is a Nps x Npg as where Npgs is 
the number of available S-BSs. 
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where XU. € (—1,0,1]. At the beginning, we determine whether the link (n,m) between the two UAV hopes 


n and m (where n Æ m), is activated in time slot t. Then mark each hop based on the communication direction 
as in Table] 


Table 2. Link states 


Value Link status 
-1 n Rx, m Tx (n receives data from m) 
0 Link is not activated 
1 n Tx, m Rx (n transmits data to m) 


Our aim here is to select the best route and scheduling solution with minimal end-to-end latency and 
BER; we can compute it as (2): 
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where L is the end-to-end latency measured in milliseconds (ms), while T; (X) is the resulting end-to-end 
measured in S-BS n when selecting the proper route and schedule it. This function is a stochastic function that 
depends on the factors: 1) location of the UAV as a S-BS, ii) interference, iii) dynamics of the queue, and iv) 
user mobility. 

Regarding the UAV location and for the sake of maintaining a better line of sight to increase the 
available communication bandwidth and downloads, each UAV is equipped with a GPS tracker that updates 
its location in a decentralized manner using a semi-distributed approach. Also, for the resulting interference 
and to minimize it, our proposed solution is utilizing the UAV to act as a flying S-BS with a reasonably high 
altitude. Adapting the UAV as an S-BS could reduce the expected interference to the lowest possible value due 
to its high altitude. 


3.1. INUSH algorithm 


Due to the high dimensionality of the routing and scheduling problem, the innovative UAV self- 
backhauling hybrid solution using RF/FSO system for 5G networks and beyond INUSH algorithm optimizes 
the transmission with capabilities and enhancement to communicate over multiple mediums and devices, where 
RF towers operated as a front-backhauling only, while the multi-hoping path is introduced via UAV S-BSs. 
These UAV S-BSs are equipped with RF and FSO communication capabilities. The decision of which of the 
available mediums to select is based on predefined factors and the measurement between every two consecutive 
hopes. 

For this kind of problems, dynamic programming techniques are used [22]. However, since we are 
dealing with a self-backhauling problem with recognized high dimensionality, such techniques are not a prac- 
tical solution. To overcome these issues, we propose the INUSH algorithm, which is a reinforcement self- 
learning algorithm that aims at minimizing the end-to-end latency while increasing the available throughput for 
everyday users. 


3.2. Semi-distributed approach 


We are using a semi-distributed approach in our proposed solution; Figure [4] shows the high-level 
architecture. The proposed architecture is designed to provide scalability and handle the increasing numbers 
of UAVs as an S-BSs and at the same time enhance the performance of the INUSH algorithm by reducing the 
time required to select the best available route with a proper propagation media. Figure [4| shows that every 
UAV S-BS is associated with a learning agent. Every learning agent is responsible for predicting the needed 
time to reach the F-BS based on recorded data from previously transmitted packets. The optimal route can then 
be calculated by taking into consideration weather turbulence, pointing error fading, and timing information 
collected by the learning agent. 

The proposed semi-distributed learning approach is proven to remove the limiting factors of the num- 
ber of used UAV as S-BSs and reducing the queuing time [4]. For the route selection process and the proposed 
algorithm, it is done in three stages, each stage runs according to a predefined algorithm and functions and runs 
in a specific order. First, the learning agent will call the INUSH algorithm which will invoke the synchroniza- 
tion control algorithm followed by the medium propagation selection algorithm. 


3.3. INUSH algorithm 


In this section, we are giving a detailed discussion of the main INUSH outlined in algorithm[1] which 
intends to be invoked by each learning agent. First, every learning agent is initializing the learning parameters 
at (line 1). At (line 3), for each time slot, the algorithm observes the network hops states, collects, and registers 
them. At (line 5), the algorithm determines the available actions. Afterward, every learning agent selects the 
optimal route using the e-greedy policy (line 6). Next, The result or action provided by the learning agent is 
forwarded as an input to the synchronization control algorithm (line 8). Finally, the output is forwarded to the 
media propagation algorithm for final evaluating. 
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Figure 4. High-level architecture of the semi-distributed learning approach 


Algorithm 1 INUSH algorithm 
1: initialize an, €n at every equipped self-learning agent 
2: for every T = 1,:::,t do 
3: measure state 5, 


4 for every learning agent n = 1l,::, Ngs do 
5: measure 5;,,; and find available actions 

6: select tn € Xn using e-greedy 

7: end for 

8 Invoke Synchronization Control Algorithm. Algp] 


9: Invoke Medium Propagation Control Algorithm. AlegB] 
10: Apply X¢ and measure the reward based on the target neighboring U AVm TRx time and its estimates E-to-E latency 
11: Update E-to-E latency to its expected value in each time slot 
12: end for 
13: Exit 


3.4. Synchronization control algorithm 

Since the proposed solution uses a semi-distribution technique, any taken decision by any of the 
learning agents may conflict with the other agent's decisions (e.g., two or more UAV S-BSs attempting to 
transmit data to the same UAV S-BS at the same time). To overcome this problem, our solution proposes 
another algorithm called the synchronization control algorithm. The aim of this algorithm is synchronizing 
the overall route selection mechanism and make sure that there is no transmission conflict between the UAVs, 
as well as reducing the interference. The pseudocode of the synchronization control algorithm is defined 
in algorithm [2] Table |3| summarizes the used parameters within this algorithm along with the meaning and 
purposes for each of them. 

From Table |3} we can see that at the beginning, the algorithm collects the current states for every 
hop in the network s+, the action selected by each learning agent xn, the selected route X; as well as the 
interfering links [,,_, (n transmits to m) that being transmitting over. This algorithm will use the information 
in Table 3] to run and return its result through three specific phases (loops) to ensure that the selected route 
is feasible, assignable, and applicable to be used by the asking UAV S-BSs. In the first phase or loop, the 
proposed solution eliminates any simultaneous activation of the selected route’s interfering links. After that, 
the second phase/loop starts, which in turn checks and makes sure that no two or more UAVs are attempting 
to transmit to the same UAV S-BS. Finally, at the third and last phase, the proposed algorithm ensures that for 
every transmitting UAV, the target neighboring UAV S-BS is in receiving mode. 
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Algorithm 2 Synchronization control algorithm 
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1: check s; and x, for n = 1; :::; NBs 
2: calculate X; 

3: for n = 1,..., Neg do 

4 if 5 4, Z 0 then 

5 


lookout for all the Trx UAV BS | € In,m knowing that 
{Si = max (Sy i)|(Kk = A Nps) U (k € Inm) 


6 Deactivate all the other Trx UAV BS k=l, k; l € In,m 
T: end if 

8: end for 

9: form = 1,:::, Ngs do 

0: if DBS Xn,m > 1 then 

1 Lookout for all the Trx UAV BS knowing that 


{S14 = maz(Sy;)|(k = 1,:: NBs) U (Xk,m = 1) 


12: Deactivate all the other Trx UAV BS k=l 
13: end if 

14: end for 

15: for n = 1,:::, Neg do 

16: if x, xm, then 

17: if sni > 5,,,; then 

18: Operate UAV BS m as a receiver BS 
19: else 

20: Operate UAV BS n as a receiver BS 
21: end if 

22: end if 

23: end for 

24: Exit 


Table 3. Synchronization control algorithm parameters 


Parameter Description 
St Current state of the network 
Zn Action selected by learning agent 
Xt Corresponding route selection 
Tom Interfering links (n transmits to m) 
Y ss Xn,m > 1 two or more UAV S-BS intended to Trx to the same UAV 
Ln—m7Lm,n make sure that every TRx UAV, the targeting neighbor UAV is in Rx mode 


3.5. Medium propagation selection algorithm 


Previous experiments and simulations show the promising advantage of hybrid systems over tradi- 
tional RF-based communication [23]. This algorithm aims to check every proposed route received from every 
learning agent for the overall bit error rate (BER) to validate the possibility of using FSO as a propagation 
medium or keeping the RF medium in use. In algorithm [3] we are assuming that the default and worst-case 
is using the RF as a propagation medium (line 1). At (line 2 and 3), the INUSH algorithm measure the atmo- 
spheric turbulence fading (Ja) as well as the pointing error fading (Jp) for every link between each UAV S-SB 


and the next UAV S-BS along the selected route. 


Algorithm 3 Medium propagation control algorithm 


1: consider using RF by default 
2: compute the atmospheric turbulence fading (Ta) at every UAV-UAV path 
3: compute the pointing error fading (Tp) at every UAV-UAV path 
4: while path not reach final backhual do 
5: compute BER; y using RF between UAV; and U AV; 1 
6: compute B ER ys. using FSO between UAV; and U AV; 41 
T: if BERfso € BER, then 
8: Switch UAV S — BS; Tx to FSO mode 
9: Switch UAV S — BS,41 Rx to FSO mode 

10: else 

11: Switch UAV S — BS; Tx to RF mode 

12: Switch UAV S — BS,,, Rx to RF mode 

13: end if 

14: end while 

15: Exit 
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Afterword and at (line 5), the INUSH algorithm will calculate the RF BER, p using the SNR value 
from the received signal strength indicator (RSSI) considering that the RF medium being used between the two 
hops being examined. Similarly, at (line 6), the INUSH algorithm will measure FSO B ER y, using the SNR 
value from the RSSI, the current weather, and the fading considering that the FSO medium is used between 
the two hops that being examined. Going ahead to (line 7), the INUSH algorithm will start to compare the 
calculated values of the BER fso and BER, ;. If the BERfso is less than or equal to the calculated B.E RF, y, 
then the FSO will be selected as a transmission medium over the RF. Simultaneously, the corresponding UAV 
S-BS; will be activated as FSO transmitting hope, and UAV S-B 5; ,; will be activated as FSO receiving hope 
(Line 8 and 9). If not and similarly, the algorithm will switch the corresponding UAV S- B5; to be transmitting 
hope using RF medium and UAV 5S- B5; , , receiving hope to RF medium as well (Line 11 and 12). The same 
process is repeated between every two hops in the selected route, pair by pair until reaching the final hope. 


3.5.1. Atmospheric turbulence model 

In FSO, the signal is affected by two significant factors: the atmospheric turbulence fading 7 = I, 
and the pointing error fading J = lp. Because of these factors, the optical signal intensity fluctuations at 
the receiver side computed as I = Ial, [24]. The INUSH algorithm determines which propagation media to 
use by finding the intensity fluctuations between each two UAV S-BSs in the selected path. To compute the 
optical signal intensity fluctuations related to the FSO channel, we are assuming the same as where the 
Gamma-Gamma distribution model is used for the link fading modeling. 

Accordingly, we can compute the probability density function (PDF) using (3): 


o8 


(ap) * Ic" 
I'(a) P(6) 
where G represents the Meijer-G function and T is the gamma function. In contrast, a and 8 are representing 


the large-scale and small-scale cells in the atmosphere. 
For o and f parameters, it is computed by (4) and (5). 


—1 
A902 
a= |exp : s 1 (4) 
(1+ 1.11a, \7/6 


fra (a) = G9» (aBI, |a. B) (3) 


and 


—1 


ela 0.510? 
(1 + 0.690;7/°)5/6 


where o? = 1.23C2(2m/X)7/°Z1"/6 is the log radiance variance, and Z is the distance between sender and 
receiver, while C? is the index of the refraction structure parameter (introduced by Kolmogorov), that constant 
for the horizontal link [25]. 


(5) 


3.52. Pointing error fading 
Relaying on the research done by Jurado-Navas et al. [26], the probability density function of Ip is 
computed by (6). 


V2 g2 
fip(Ip) = Alp OS Ip < Ao (6) 
0 


where Ao is the fraction of the collected power when r = 0; r and w; = w[1 + €(AZ/mwg)?]!/? are the hole 
radius and the beam waist at distance Z. 

The most important parameter in (sh is 0 = €z¢q/(205) which is the relative ratio between the beam 
radius and the pointing error standard deviation. In the same manner, if à = 1, then the pointing error is too 
high. Hence, if ? is going high, the pointing error itself will go down while (If 0 — oo, the pointing error 
becomes negligible). For the sake of minimizing the pointing error fading, and as we mentioned before; each 
UAV S-BS will be equipped with a GPS tracker, which allows the UAV S-BSs to update their location in the 
corresponding locations manager. From the corresponding locations manager, every UAV will know where the 


other UAVs’ location will be at any time instance and align its FSO receivers or transmitters accordingly. 
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3.5.3. Mixing fading parameters 

In our solution, we are assuming that all links in the selected path or route are independent of each 
other since the date transmission between every two hops in the same route does not depend on any previous 
hops at the same route, and since all the previous equations that used to calculate the end-to-end relation and 
measure the SNR value to calculate the BER between the different hops in the selected route is a little bit 
complicated, so we are going to use a simplified expression (7). 


A12 
Àfso = e min(M,A 7 
f ros ERE) (A1, A2) (7) 
where A, and A» is the BER of the FSO and RF in the selected path. Using the same approach, if A3 and A4 is 


the BER of FSO and RF links, we can compute the BER of that link (P5) as (8): 


A3A4 


a CEs VEE 


~ mán(Aa, À4) (8) 
on the next hop where the next UAV S-SBs is intended to receive the signal, the propagation media selection 
algorithm will select the link with the lowest BER Asc = maz(A so, Arf). 


4. RESULTS AND DISCUSSION 
4.1. INUSH complexity analysis 

In this section, we analyze the complexity of the proposed INUSH algorithm for one iteration. We 
considered the measured network size, where the number of UAV S-BSs is (N gs) is the deciding factor in the 
network. We will also show how the INUSH algorithm's complexity is significantly less than the traditional 
reinforcement learning algorithms such as SARSA and SCAROS [4]. 

To determine the big-O complexity of the INUSH algorithm, we will begin by calculating the com- 
plexity of the synchronization control and media propagation control algorithms. Then, the complexity for both 
algorithms will be combined to build the total INUSH algorithm complexity. We will start by analyzing the 
synchronization control algorithm. As we discussed before, it consists of three phases, and each phase is com- 
posed of one for-loop, which is responsible for checking the status and the selected action of the neighboring 
hops. All three loops are running independently and in sequence. The worst-case complexity for all the phases 
is equal to O(3N2 s), which is equal to O(N? s) since these operations have to find the maximum values of 
the state of all available S-BS which equivalent to O(N gs). And as this keep repeating Npgs times, so the 
complexity for each phase is O(NZ s), and since there are three independent loops so the overall complexity 
remains O(N o) 

Going to the medium propagation control function, it is clear that the most computationally extensive 
sequence is within the while statement(lines 4-14). This while statement represents the number of hops in 
the selected route, denoted by Mgs, so the overall complexity of the medium propagation control function 
is O(Mps — 1), which is the number links that the algorithm will examine until reaching the core network. 
Using the same approach, we will evaluate the complexity of the INUSH algorithm itself. We noticed that the 
most complicated operations performed inside the loop (lines 4-7) in which the synchronization and the media 
propagation functions were invoked. Hence, we are using the e-greedy policy; the worst-case is finding the 
required action that leads to the minimum latency. The time complexity for this operation can go as O(Ngs), 
which is the same as e-greed big-O complexity. Since the operation is run independently inside each learning 
agent, the complexity will remain the same. 

Regarding synchronization control function and as discussed in the previous paragraph, it is growing 
up to O(N2 s). Now, for the media propagation selection function and similarly, as discussed before, it goes 
up to O(Mpgs) in the worst case as well. Using the same analysis technique, Ortiz ef al. computed the 
complexity of Q-learning and SARSA algorithms. Comparing our proposed solution to their work, we can see 
that their complexities depend firmly on the network size |x| due to the use of a centralized learning technique 
to solve the formulated problem. Therefore, as TNs x N2 s X Megs, INUSH shows significantly smaller 


n=1 
time complexity compared to the other learning schemes. 


4.2. Reliability against weather turbulence 
In this section, we are discussing the reliability and performance of our proposed solution under dif- 
ferent weather turbulence conditions. During the simulation, and as this paper targets 5G and beyond networks, 
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we choose to do the simulation for sub-6 GHz frequencies. The aims of the performed simulation to give an 
insight into how the INUSH algorithm will work using these frequencie bands. For the used parameters in our 
simulation, Table fļis listing them. 

For the number of hops, our proposed solution was designed and evaluated for N number of hopes. 
For the sake of performing the simulation, we choose 20 as a value of N. The reason behinds this selec- 
tion that the 20-hops may cover an area of 400 m? (20 x 20 = 400 m°) in the case of approaching ultra- 
density deployment. Similarly in the medium density deployment, the covered area could reach up to 3-km 
(150 x 20 = 3000 m?) and in sparse this value becomes 10-km (500 x 20 = 10000 m?). 


Table 4. Simulation parameters 


Parameter Value Description 
Noise floor -90 dBm RF noise floor 
Number of Hops 0-20 Number of hops to reach the core 
RF transmitted power -10 dBm RF transmitted power 
1-20m Distance in ultar-density deployment 
Distance between hops 1-150 m The HPC2N Seth log 
1-500 m Distance in sparse deployment 


Regarding the SNR for every link, we can calculated as (9), 


SNR = Receivedpower — Noise ploor (9) 


while the received power calculated using Friis transmission as (10), 


Pra = Ptz + Gta + Gra + 20L0g10(Lamda/(4PiDr)) (10) 


where Pr is the received power, Ptz is the transmitter power, Gtz is the transmitter gain, Grz is the receiver 
gain, Lamda is the wave length, and Dr is the distance between the transmitter and the receiver. 

Going further in this discussion, we performed the simulation using different weather turbulence con- 
ditions. Table[5]summarizes the considered weather conditions along with their related parameters. For the sake 
of the performed experiment more realistic, we set the weather fluctuation percentage to be + 10% between 
every hop and the next one. 


Table 5. Weather conditions 


Weather Turbulence Description 
Good weather Visual clearance is around 75% 
Moderate weather Visual clearance is around 5596 
Bad weather Visual clearance is around 2596 


We considered different values to simulate the different weather conditions in our experiments; 
Figure [5| shows these values. To add more realism, we set the weather fluctuation factor to + 10% between 
every two hops. We choose 3-different values as the following, Figure 5(a) shows the expected simulation 
values during summer where the visibility clearance is around 75%. For the autumn and spring seasons, 
Figure 5(b) simulates them with around 55% visibility clearance, and finally, Figure 5(c) simulates the winter 
with around 25% visibility clearance. During this simulation, we took into consideration the distance between 
different hops as a parameter. This parameter has a significant effect on the transmission for both the RF and 
FSO communication and the weather fluctuation that we mentioned before, which has a considerable effect on 
the FSO transmission. 


4.2.1. INUSH simulation summary 

Subsection and shows the reliability against different weather turbulence. It 
represented one experiment of 20 hops under 25% as bad weather, 55% as moderate weather, and 75% as good 
weather conditions. We repeated the experiment of 20 hops for 100 times and summarized the average BER at 
different weather and densities in Table [6] The selection ratio between FSO/RF is presented in the rightmost 
column (H-Ratio). 
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Figure 5. Weather turbulence fluctuation (a) good weather turbulence, (b) typical weather turbulence, and 
(c) bad weather turbulence 
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m) 


Visibility Deployment RF Path BER FSO Path BER Hybrid Path BER H-Ratio (RF:FSO) 

25% Dense 1.27 x 10-5 2.90 x 10702 1.21 x 1075 0.99 : 0.01 
Medium density 4.34 x 107? 2.77 x 10-9? 6.90 x 1078 0.54 : 0.46 

Sparse 2.42 x 107} 3.21 x 1070? 2.14 x 107? 0.22 : 0.78 

45% Dense 7.48 x 1076 1.41 x 10-9 1.34 x 1077 0.73 : 0.27 
Medium density 4.44 x 107? 1.27 x 1076 1.01 x 10-6 0.16 : 0.84 

Sparse 2.32x 107} 1.38 x 10-9 1.24 x 10-9 0.06 : 0.94 

5596 Dense 1.02 x 10-5 9.06 x 10-? 3.54 x 10-? 0.40 : 0.60 
Medium density 4.14 x 107? 8.82 x 10-9 7.89 x 1079 0.07 : 0.93 

Sparse 2.38 x 107! 9.40 x 10-? 9.14 x 10-9 0.02 : 0.98 

7596 Dense 6.54x 1076 4.11 x 10-1? 4.10 x 10-13 0.00 : 1.00 
Medium density 4.37 x 107? — 4.04 x 10-13 4.04 x 10-13 0.00 : 1.00 

Sparse 2.27 x 10-1 3.78 x 10-12 3.78 x 10-13 0.00 : 1.00 
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4.2.2. Reliability under good weather turbulence 


As we mentioned before, good weather turbulence occurs when the visual clearance is around 75%. 
In our simulation, we assumed that it needs up to 20 hops to reach the backhauling base stations (F-BS). Our 
simulation is done using sub-6 GHz frequency bands. 

Figure |6]shows how INUSH behaves under different network densities, namely sparse, medium den- 
sity, and Dense Network using 6 GHz frequency bands. Figure 6(a) illustrates the communication behavior in 
a sparse network where the distances between any two hops in the selected route does not exceed 500 meters. 
Here we can see that our hybrid solution takes the same behavior as the FSO-Only solution, which meets our 
expectation as the visual clearance is around 75% which gives an advantage of using FSO communication. 
Using the same approach, Figure 6(b) shows the result in the medium-density networks, where the distances 
between any two hops are up to 150 meters. The same as Figure 6(a), our hybrid solution takes the same 
behavior as the FSO-Only solution. Finally, regarding Figure 6(c), it shows the result in the dense network 
deployment, where the distances between any two hops are not exceeding 20 meters. When comparing the 
BER between the three route types, RF-Only, FSO-Only, and hybrid route, we can notice, as in Figure[7] the 
BER is evaluated under different network densities; sparse in Figure 7(a), medium density in Figure 7(b), and 
dense in Figure 7(c). Since the weather condition is in its best condition, where the clearance is around 75%, 
the hybrid solution rely mainly on FSO links due to the lower BER, hence the same BER as the FSO-only 
solution in Figures 7(a)-(c). 


4.2.3. Reliability under bad weather turbulence 


After discussing the result under good weather turbulence, we will discuss INUSH's performance and 
behavior under bad weather turbulence. As we mentioned before, bad weather turbulence occurs when the 
environment visual clarity around 25% or less. For the number of hops, we will maintain the same assumption 
as before, which is up to 20 hops until reaching the corresponding F-BS. Figure[B|shows how our proposed al- 
gorithm behaves under different network densities: sparse, medium and dense network using 6 GHz frequency 
bands. 

Figure 8(a) shows the communication behavior in sparse network deployment where the distances 
between any two hops are up to 500 meters. We can notice here that our INUSH hybrid solution takes better 
behavior comparing to both RF-Only and FSO-Only solutions. Using the same approach, Figure 8(b) shows 
the result for the medium density networks (distances between any two hops are up to 150 meters), The same as 
Figure 8(a), INUSH hybrid solution gives better BER values comparing to RF-Only and FSO-Only solutions. 
Finally, Figure 8(c) shows the result in dense network topology (distances between any two hops not more than 
20 meters), which give the same behavior as the RF-Only solution due to the short communication range. 

In Figure ] we present the measured BER between these three route types (RF-only, FSO-only, and 
hybrid routes) in sparse deployment as in Figure 9(a), medium density as in Figure 9(b), and dense deployment 
as in Figure 9(c). And since we discuss the behavior under the bad weather (the visual clearance is around 
25%), the INUSH hybrid solution will rely heavily on FSO-only when the network is sparse due to the high 
BER in RF in such condition as seen in Figure 9(a). When the network is densely deployed, RF channel will 
have a very low BER and the hybrid solution will rely on RF-only links as shown in Figure 9(c). 
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Figure 6. 6 GHz network density deployment comparison under good weather (a) sparse network, (b) medium 
density network, and (c) dense network 
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Figure 7. 6 GHz network density deployment BER comparison under good weather, the number in hybrid bar 
is the ratio between RF/FSO (a) sparse network BER, (b) medium density network BER, and (c) dense 
network BER 
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Figure 8. 6 GHz network density deployment comparison under bad weather (a) sparse network, (b) medium 
density network, and (c) dense network 
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Figure 9. 6 GHz network density deployment BER comparison under bad weather, the number in hybrid bar is 
the ratio between RF/FSO (a) sparse network BER, (b) medium density network BER, and (c) dense network 
BER 


4.2.4. Reliability under moderate weather turbulence 

After discussing the extreme weather cases, which weather either in its best or worst condition, we 
will discuss the INUSH algorithm under average weather turbulence. In moderate weather turbulence, the 
visual clarity is 5596, representing the average during the year as it is present in almost two seasons (spring 
and autumn). As mentioned before, moderate weather turbulence occurs when the environment visual clarity 
around 5546. 

In our experiment, we considered a light fluctuation in weather turbulence between every hop and next 
consecutive hop on the selected route. We maintain the same value we used in the previous simulation to be 
+10% of the current weather condition at the last examined hop. Such fluctuation happened for many reasons 
such as temporary smoke or temporary blockage, even any type of FSO signal attenuation that could eventually 
affect receiving the FSO signal. For the number of hops, we maintained the same assumption as before, which 
is up to 20 hops until reaching the corresponding F-BS. Now we are going to discuss the result when using 
6 GHz frequency; Figure [I0] shows how our proposed algorithm behaves under the different network density 
using 6 GHz frequency bands. 

Figure 10(a) shows the communication behavior under a sparse network deployment where the dis- 
tances between any two hops are up to 500 meters. It is noticeable that the INUSH hybrid algorithm shows 
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almost the same behavior as the FSO-Only solution, and the reason behind that the mm-Wave communication 
range is relatively low. Using the same approach as in a sparse network, Figure 10(b) shows the obtained result 
in the medium-density networks (distances between any two hops are up to 150 meters). In this type of deploy- 
ment, our INUSH hybrid algorithm took the same behavior as the FSO-Only. However, still, there are a small 
number of hops that decided to communicate over RF channels. Finally, Figure 10(c), shows the obtained result 
in the dense network deployment (distances between any two hops are not more than 20 meters). Here we can 
see that at some points, the solution decided to go over RF channels since the BER at that point is smaller than 
the FSO BER due to the short observed distance, which gives an advantage to the RF communication. 
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Figure 10. 6 GHz network density deployment comparison under moderate weather (a) sparse network, 
(b) medium density network, and (c) dense network 


By comparing the BER between the three route types (RF-Only, FSO-Only, and Hybrid routes) in 
the different network densities, Figure [11] shows the differences between them. For example, Figure 11(a) 
represent the average BER in the sparse network and showing how the INUSH performed the same as FSO- 
Only techniques. Figure 11(b), comparing the obtained BER in the medium density network between RF, FSO, 
and our hybrid technique. We can notice that the result is better than both RF and FSO only solution since 
it chooses the lowest BER value between the RF and the FSO, resulting in better route performance Finally, 
Figure 11(c), shows the obtained BER in the dense network. We can see that it is much better in the hybrid 
solution than the RF and FSO only solutions since the hybrid solution chooses the lowest BER between the RF 
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and the FSO, resulting in better QoS. To clarify the obtained result, Table[7] give more details and summarization 
of Figure[11] which comparing our INUSH hybrid algorithm against RF-Only and FSO-Only solutions. 
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Figure 11. 6 GHz network density deployment BER comparison under moderate weather, the number in 
hybrid bar is the ratio between RF/FSO (a) sparse network BER, (b) medium density network BER, and 
(c) dense network BER 


Table 7. Routes BER comparison based on network density 
Route Type Sparse Medium Density Dense 


RF:FSO hops 0:20 2:18 7:13 
RF BER 1071 10-15 10755 
FSO BER 1078 1078 1078-5 

Hybrid BER 1078 1079 1078-9 


4.3. Robustness against load imbalance 

In this part, we discuss the load-balancing in self-backhauling networks. Figure [12] shows how the 
load imbalance will affect the throughput. While Figure[I3|shows the same for end-to-end latency. As we can 
see in Figure[12] INUSH shows significant throughput increases (up to 8-Gbps) compared to SCAROS, which 
is less than 8-Gbps and RT which is steady of almost 1.75-Gbps while MTFS with 3-Gbps INUSH algorithm 
takes the better values between the FSO and the RF. For end-to-end latency, as in Figure [I3] INUSH shows a 
steady line with almost 0.055 seconds, while SCAROS shows a higher latency with 0.07 seconds. For RT, we 
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can see that it shows almost 0.08 seconds over the same period while MTFS increasingly shows high end-to- 
end latency with up to 0.25 seconds over the same period and the same as before; INUSH chooses the route 
with the smaller BER, which affects the overall performance. 
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Figure 12. Throughput comparison between different schemes 
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Figure 13. End-to-End latency comparison between different schemes 


5. CONCLUSION 

In this work, we presented a scalable and robust algorithm, INUSH, which depends on decentralized 
self-learning approach to find the optimal route in self-backhauled mm-Wave networks with a realistic network 
dynamics nature such as interference and mobility. Our proposed solution uses UAVs as self-backhauling base 
stations while keeping the standard stationary macro, micro, and small towers to provide front-backhauling 
functionality to the mobile users. Using this approach, we increased the available bandwidth, throughput, and 
data rate for normal mobile users because of utilizing the stationary base stations for them. 

Since the UAV intended to work as S-BS, it will reduce the blockage percentage to a minimum as 
well as the interference and end-to-end latency. We also optimize the UAV S-BS to use the FSO as media 
propagation between every two hops whenever it is possible along the selected route, which in turn increase the 
backhauling capacity, reduce the overall route interference, and increase the SNR. In this paper, we proposed 
a hybrid solution to use one of these technology (FSO or RF), and we assumed that FSO channel fading is 
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Gamma-Gamma in the presence of the pointing error. The initial evaluation shows that the INUSH algorithm 
shows a significant enhancement of the traditional deep learning approaches. 
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